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EXISTENCE RESULT FOR A CLASS OF QUASILINEAR ELLIPTIC 
EQUATIONS WITH (p-q)-LAPLACIAN AND VANISHING POTENTIALS 

M. J. ALVES, R. B. ASSUNgAO, AND O. H. MIYAGAKI 


Abstract. The main purpose of this paper is to establish the existence of positive solutions 
to a class of quasilinear elliptic equations involving the (p-g)-Laplacian operator. We consider a 
nonlinearity that can be subcritical at infinity and supercritical at the origin; we also consider 
potential functions that can vanish at infinity. The approach is based on variational arguments 
dealing with the mountain-pass lemma and an adaptation of the penalization method. In order 
to overcome the lack of compactness we modify the original problem and the associated energy 
functional. Finally, to show that the solution of the modified problem is also a solution of the 
original problem we use an estimate obtained by the Moser iteration scheme. 


1. Introduction and main result 


In this paper we consider a class of quasilinear elliptic equations involving the (p-g)-Laplacian 
operator of the form 

j —ApU — AqU + a{x) u + b{x) u = f{u), x G 

\ u{x) >0, we n X e ^ ^ 

The m-laplacian operator Amu{x) is dehned by 

Amu{x) = div(| Vm(x)|™'“^Vm(x)), 


for m G {p, g}, where 2 ^ q ^ p < N. The Sobolev space is dehned by 

^ g {du/dxi){x) G 

and the critical Sobolev exponent is given by m* = Nm/{N — m), also for m G {p, q}. 

The nonlinearity /: M —)■ M is a continuous and nonnegative function that is not a pure 
power and can be subcritical at inhnity and supercritical at the origin. More precisely, the 
following set of hypotheses on the nonlinearity / is used. 

(/i) \imsup^_^Q+ sf{s)/sP* < +CX). 

(/ 2 ) There exists r G {p,p*) such that hmsup^^_,_oo s/(s)/s’’ = 0- 

(/s) There exists 9 > p such that 0 ^ 9F{s) ^ s/(s) for every s G M+, where we use the 
notation F{s) = J f{t) df. 
ih) fit) = 0 for every t ^ 0. 

The following properties are easily seen: under hypothesis (/i) there exists Ci G such that 
|s/(s)| ^ Ci|s|^* for s close to zero; and under hypothesis (/ 2 ) there exists C 2 G M+ such that 
|s/(s)| ^ C 2 |s|’' for s large enough. Combining these results and dehning cq = max{ci,C 2 }, we 
have the pair of inequalities 

|s/(s)| ^ co|s|^ and |s/(s)| ^ colsl"^ (s G M). (2) 


Is is worth noticing that hypothesis (/a) extends a well known condition which was hrst 
formulated by Ambrosetti and Rabinowitz [5]. It states a sufficient condition to ensure that 
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the energy funcional, associated in a natural way to this type of problem, verifies the Palais- 
Smale condition. Recall that a functional J: —)■ M is said to verify the Palais- 

Smale condition at the level c if any sequence {un )nen C such that J{un) —t c and 

J'{un) —)■ 0, as n —)■ +CX 0 , possess a convergent subsequence. Hypothesis (/s) also allows us to 
study the asymptotic behavior of the solution to the problem. 

As an example of a nonlinearity / verifying the above set of hypotheses, for a > p* and for 
r G {p,p*) given in hypothesis (/ 2 ), we define 

if 0 ^ f ^ 1 ; 
if 1 ^ f. 


f{t) = 

I ^ 

We also assume that the functions a, b: 



—)■ JR are continuous and nonnegative. Moreover, 
the following set of hypotheses on the potential functions a and b is used. 

(Pi) a G L^/P{R^) and b G 

(P 2 ) a(x) ^ ttoo and b{x) ^ b^o for every x G Pi(0), where a^o, boo ^ are positive constants 
and Pi(0) denotes the unitary ball centered at the origin. 

(P 3 ) There exist constants A G M"*" and Pq > 1 such that 

— 2 ^—-T inf \x\P^^^P~^'^a(x) ^ A. 

As an example of a potential function a verifying this set of hypotheses, for A G and 
Po > 1 given in hypothesis (P 3 ) we define 


{ 0 , if |a;| ^ Po — 1; 

- Po + 1), if Po - 1 < |x| < Po; 

A|x|-p'/(P-b, ifPo^|x|. 

An example of a potential function b can be obtained in a similar way with minor modifications. 

The (p-q')-Laplacian operator generalizes several types of problems. For example, in the case 
2 = q = p with a{x) = b{x) = V{x) and f{u) = 2g{u), problem (1) can be written in the 
form —Am + V{x)u = g{u), which appears in the study of stationary solutions of Schrodinger 
equation and has been extensively studied by several authors; and in the case 2 ^ q = p with 
a{x) = b{x) = —V{x) and f{u) = 0, problem (1) assumes the form of the eigenvalue problem 
—ApU = V{x)\u\P~‘^u. 

The interest in the study of this type of problem is twofold. On the one hand we have the 
physical motivations, since the quasilinear operator {p-q)- Laplacian has been used to model 
steady-state solutions of react ion-diffusion problems arising in biophysics, in plasma physics 
and in the study of chemical reactions. More precisely, the prototype for these models can be 
written in the form 


Ut = — div[P(M)VM] -|- f{x, u), 

where D{u) = ap\Vu\P~'^+ bg\Vu\'^~‘^ and ap,bq G M"*" are positive constants. In this framework, 
the function u generally stands for a concentration, the term div[P(M)VM] corresponds to the 
diffusion with coefficient D{u), and f{x,u) is the reaction term related to source and loss pro¬ 
cesses. See Cherfils and Il’yasov [19], Figueiredo [25, 26], Benouhiba and Belyacine [14], Mercuri 
and Squassina [30], Wu and Yang [40], Yin and Yang [41], Chaves, Ercole and Miyagaki [17, 18], 
and references therein for more details. In addition, a model of elementary particle physics was 
studied by Benci, D’Avenia, Fortunato and Pisani [11] which yields an equation of the same 
class as that in problem ( 1 ). 

On the other hand we have the purely mathematical interest in these type of problems, 
mainly regarding the existence of nonnegative nontrivial solutions as well as multiplicity results. 
In what follows we present a very brief historical sketch to show some hypotheses on the 
nonlinearity that have been used by several authors in recent years as sufficient conditions to 
guarantee the existence of solutions. 
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We begin by considering the case 2 ^ q = p < p*, which includes both the Laplacian operator 
with p = 2 or the p-Laplacian operator with p > 2; we also mention some papers dealing with 
bounded domains and others dealing with the entire space 

Berestycki and Lions [15] considered a positive, constant potential function to show an ex¬ 
istence result. Coti Zelati and Rabinowitz [21], Pankov [32], Pankov and Pfluger [33], and 
Kryszewski and Szulkin [28] considered periodic potential functions with a positive inhmum. 
Zhu and Yang [42, 44] assumed that the potential is asymptotic to a positive constant. Alves, 
Carriao and Miyagaki [2] studied a problem involving an asymptotically periodic potential. The 
case of a coercive potential was treated, among others, by Costa [20] and Miyagaki [31]. For a 
weakened coercivity condition we refer the reader to Bartsch and Wang [9]. The case of radially 
symmetric potentials were considered by Alves, de Morals Filho and Souto [3] and Su, Wang 
and Willem [38], where these authors established some embedding results of weighted Sobolev 
spaces to obtain ground state solutions. Rabinowitz [35] introduced a hypothesis where the 
limit inferior of the potential outside a bounded domain is strictly greater than its inhmum on 
the whole space. Afterwards, del Pino and Felmer [22] weakened this condition by considering 
a situation where the minimum of the potential on the boundary of an open bounded set is 
strictly greater than its minimum on the closure of this set. The case of sign-changing poten¬ 
tials related to singular perturbation problems were considered by Ding and Szulkin [24] and 
by Alves, Assungao, Carriao and Miyagaki [1]. 

As we have seen, most of the papers cited assume that the potential is positive at inhnity. 
However, the case where the potential can vanish at inhnity was also studied, among others, by 
Berestycki and Lions [15], Yang and Zhu [43], Benci, Grisanti and Micheletti [12], Ambrosetti 
and Wang [7], Ambrosetti, Felli and Malchiodi [6], Alves and Souto [4], and Bastos, Miyagaki 
and Vieira [10]. 

In problem (1) we consider the exponents 2 ^ q ^ p < N and we allow the particular 
conditions hminf| 3 ;|^_|_oo = 0 and liminf| 2 ,|_j._|_oo = 0, called the zero mass cases. These 

constitute the main features of our work. 

Our result reads as follows. 

Theorem 1.1. Consider 2 ^ q ^ p < N and suppose that the potential functions a and 
b verify the hypotheses {Pi), (A) o,nd (P3) and that the nonlinearity f verifies the hypothe¬ 
ses (/i), (/ 2 ), (/a), and {fi}- Then there exists a constant A* = A*{aoo,boo,d,T,Co) such that 
problem (1) has a positive solution for every A ^ A*. 

Usually, a solution to problem (1) is obtained as a critical point of the corresponding energy 
functional dehned in some appropriate Sobolev space. To do this one uses critical point the¬ 
ory, mainly of minimax type; see Mawhin and Willem [29], Struwe [37], and Willem [39]. A 
well known result concerning the existence of a nontrivial weak solution is that if the energy 
functional verihes the geometry of the mountain-pass lemma near the origin and also verihes 
the Palais-Smale condition, then problem (1) has at least one solution. The main difhculty 
in proving the existence of solution to problem (1) resides in the fact that the embedding of 
the Sobolev space in the Lebesgue space is not compact due to 

the action of a group of homoteties and translations. Besides, the Palais-Smale condition for 
the corresponding energy functional cannot be obtained directly. Adding to these difficulties, 
we have to consider the presence of both operators ApU and AgU. When q < p the study of 
problem (1) does not allow the use of the Lagrange’s multipliers method due to the lack of ho¬ 
mogeneity; moreover, the first eigenvalue of the —ApU operator brings no valuable information 
on the eigenvalue of the —AgU operator; finally, the method of sub- and super-solutions cannot 
be applied. Therefore, to study problem (1) we are required to make a careful analysis of the 
energy level of the Palais-Smale sequences in order to obtain their boundedness and also to 
overcome the lack of compactness. Furthermore, we have to adapt the Moser iteration scheme 
to our setting, since this is a crucial step to obtain an estimate for the solution. 
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Inspired mainly by Wu and Yang [40] regarding the (p-g)-Laplacian type operator, and by 
Alves and Souto [4], with respect to the set of hypotheses, we adapt the penalization method 
developed by del Pino and Felmer [22] to show our existence result. The basic idea can be 
described in the following way. In section 2 we modify the original problem and study its 
corresponding energy functional, showing that it verifies the geometry of the mountain-pass 
lemma and that every Palais-Smale sequence is bounded in an appropriate Sobolev space. 
Using the standard theory this implies that the modified problem has a solution. In section 3 
we show, using the Moser iteration scheme, that the solution of the auxiliary problem verifies 
an estimate involving the norm. Finally, in section 4 we use this estimate to show 

that the solution of the modified problem is also a solution of the original problem (1). 


2. An auxiliary problem 


In order to prove the existence of a positive solution to problem (1) we establish a variational 
setting and apply the mountain-pass lemma. Using hypothesis (Pi) we dehne the space 

P = e n : f a(a:)|M|^dx <-1-00 and [ 6 (a:) dx <-|-oo|, 

*- JrN ) 

which can be endowed with the norm ||m|| = ||m||ip+ ||R|lig, where we denote 

i/p 


and 


1^11 l,p — 




|VM|^da;-|- / a{x)\u\^dx 


|VM|'^da;-|- / h{x)\u\‘^ dx 


1/4 


The Euler-Lagrange energy functional / : P ^ M associated to problem (1) is defined by 


/(m) = - [ |VM|^da;-l-- [ a(x)\uf dx 
P Jr^ P Jr^ 

+ - [ |Vm|Mx + - [ b{x)\u\^dx 
Q Jrn Q Jrn 


F{u) dx. 


Using the hypotheses on the nonlinearity / we can deduce that I G moreover, for 

every u,v & E its Gateaux derivative can be computed by 


I'(u)v = 


\Vu\^ "^Vu - Vvdx + / a{x)\u\^ '^uvdx 


+ / |Vm|^ ^VM-Vnda;-|- / b{x)\u\'^ ‘^uv dx — / f{u)vdx. 

Jrn Jrn Jrn 

It is a well known fact that if m is a critical point of the energy functional /, then m is a weak 
solution to problem (1). This means that 

/ I ■ V0dx-I- / a{x)\u\^~'^u4> dx 


+ / V 0 dx+ / b{x)\u\‘^-^u4>dx 

Jr^ Jr^ 


f{u)(pdx = 0 


for every n G P. 

Now we define the energy functional loo - n Po’^(Pi(0)) ^ M by 

Ioo{u) = - [ iVwl^dx-l— [ aoo\u\^ dx 

P Jbi{o) P Jbi{o) 

+ - I dx-I-- j boo\u\'^dx— I E{u)dx. 

0. Jbi{o) Q Jbi{o) Jbi{o) 
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Using the hypotheses (Pi) and (P2) it can be shown that it is well dehned. Our hrst lemma 
concerns the geometry of this functional. 


Lemma 2.1. The functional loo verifies the geometry of the mountain-pass lemma. More 
precisely, the following claims are valid. 

(1) There exist tq, /tq ^ such that Ioo{u) ^ To ll'^ll = ''"o- 

(2) There exists Cq G [P^’^(Pi( 0)) fl P^’'^(Pi(0))] \{0} such that ||eo|| ^ vq and /oo(eo) < 0. 

Proof. By using the hypotheses (/i), (/2), and (/s) it is standard to verify item (1). 

By hypothesis (/a) it follows that there exist 9 > p and Cg G such that P(s) ^ Cg |s|^. 
Now, if M G [Po’^(Pi(0)) n Po’''(5i(0))] \{0}, then 

Ioo{tu) ^ - \t\^ [ iVtil^da; + — |f|^ [ l^l^da; 

P Jbi{o) P Jbi{o) 

+ “ [ IVm|'^ dx + — |f|'^ [ l^l'^dx — Co |f|^ [ |m|^ dx. 

*? Jbi{0) 9 Jbi{0) Jbi{0) 

Using this inequality we deduce that there exist tu G M’*' large enough such that, taking eg = tuU, 
we have ||eo|| ^ rg and /oo(eo) < 0. This concludes the proof of item (2). □ 


We denote by d the mountain-pass level associated to the functional loo, that is. 


d = inf max looilit)), 
76r tG[0,l] 


where 


r = {7 G ^([0,1]; pi’^(Pi(0)) n D^’fiBfiO))) : 7(0) = 0 and 7(1) = eo} 

and the function eg G [P^’P(Pi( 0)) fl P^’^(Pi( 0))]\{0} is given in Lemma 2.1. It is standard to 
verify that the mountain-pass level d depends only on Qoo, on boo, on 9, and on the function /. 

For P > 1 and for 9 > p given in hypothesis {fs), we set k = 9p/{9 — p) > p and we dehne a 
new nonlinearity g: x M —)■ M by 




k 


if |x| ^ P or if |x| > P and f{t) ^ 
\t\P-H, if |x| > P and f{t) > ^ \t\P-H. 


ax 


k 


\trH- 


Using the notation G(x, t)= g{x, s) ds, by direct computations we get the set of inequalities 


g{x,t) ^ 1^1^ 1^1 ^ 


G{x,t) = F{t), 


if |x| ^ P; 


G{x,t) ^ \t\P H, if |x| > P > 1. 
kp 


Now we dehne the auxiliary problem 

—ApU — AqU -\- a{x) u -I- h{x) 


u 




u = g{x,u), 


u{x) >0, ue n 


X G 
X G 


pN. 


pN 


( 3 ) 

( 4 ) 

( 5 ) 

( 6 ) 


The Euler-Lagrange energy functional J: P —)■ M associated to the auxiliary problem (6) is 
given by 


J{u) = - \Vu\Pdx + - 
PP 

^ ' |Vm|Mx+ ^ 


9 Jm.^ 


a{x)\u\P dx 

6(x)|M|^dx— / G{x,u)dx. 
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Using the hypotheses on the nonlinearity / and on the potential functions a and b we can 
show that J G C'^(-E;R); moreover, for every u,v & E its Gateaux derivative can be computed 
by 


J'{u)v= / |Vm|^ ‘^Vu-Vvdx+ / a{x)\uf ‘^uvdx 


+ / iVul*^ ^VM-Vndx+ / b{x)\u\'^ “^uvdx— / g{x,u)vdx. 


As before, critical points of the energy functional J are weak solutions to problem ( 6 ). 

Our next goal is to apply the mountain-pass lemma to show that problem ( 6 ) has a positive 
solution. 


Lemma 2.2. The functional J verifies the geometry of the mountain-pass lemma. More pre¬ 
cisely, the following claims are valid. 

(1) There exist ri, pi G M"*" such that J{u) ^ pi for ||n|| = ri. 

(2) There exists Ci G 0 \{0} such that ||ei|| ^ ri and J(ei) < 0. 


Proof. Using the equality (4) and the inequality (5) together with the hypotheses (/i) and (/a) 
and the hrst inequality in ( 2 ), we obtain 

f a(x) \uf’ 


J{u) ^ ^ ||m|IL + y 


p ■■ ' q 




F{u) dx 




kp 


dx 


IIP I ^11 \\q ^0 

^ “ 1p + - p L - -p 

p q 0 


\u\ 




p kp J " q 

Now we apply the Sobolev inequality 


\u\ 


\uf dx 


Co I IP* 

— \u\ 


kp 


\u\ 


Pq Q miLP* • 


lx 


\u\ 




^Sm iVupdx for allu G {me{p,q}) 


(7) 


p*/p 


in the computations above and set S = maxlAp, Sq} to get 

A«) > (i - ^) ll<, u NIL - X'(/jv.r d. 


,1 11 

^ mm <--—, - 

p kp q 


p*/p 


If we take ||n||a ^ and ||m||^ ^ small enough, it follows that ||n||i p and ||n||^ ^ are also small enough. 
For that reason, we obtain the existence of ri, pi G M"*" such that J{u) ^ qii for H-uH = ri. This 
concludes the proof of item ( 1 ). 

By dehnition we have that G{x,u) = F{u) for all u G fl \{0}. 

Arguing as in the proof of Lemma 2.1 we conclude that there exist ri,f„ G such that 
Cl = tuU verify the inequalities ||ei|| ^ ri and J(ei) < 0. This concludes the proof of item (2). 
The lemma is proved. □ 


Since the functional J has the geometry of the mountain-pass lemma, using Willem [39, 
Theorem 1.15] we obtain a Palais-Smale sequence {un)neN C E such that J{un) c and 
J'iun) —)■ 0 as n ^ -foo. Here c G M+ is the mountain-pass level associated to the energy 
functional J, that is, 

c = inf max Jfyft)), 

76r tG[0,l] 


where 


T = {7 G G([0,1]; G : 7 ( 0 ) = 0 and 7 ( 1 ) = ei} 
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and Cl G [D^’P{Bi{0)) fl D^’^(i?i(0))] \{0} is the same function verifying inequality J{ei) < 0 
in Lema 2.2. Using the hypothesis (/4), without loss of generality we can suppose that the 
sequence {un)nm C E consists of nonnegative functions. 

We note that for all u G [U)^’^(i?i(0)) fl U)^’'?(i?i(0))] \{0} the inequality J{u) ^ Ioo{u) is 
valid, and this implies that 

c ^ d. (8) 

Now we prove the boundedness of the Palais-Smale sequences for the functional J. 


Lemma 2.3. Suppose that the potential functions a, b verify the hypothesis {Pi), and that the 
nonlinearity f verifies the hypotheses {fi), {f 2 ), {fs), o,nd {f^)- //’ (wn)neN C E is a Palais- 
Smale sequence for the energy functional J, then the sequence {un)nm ^ E is bounded in E. 


Proof. To obtain our thesis it is sufficient to prove that both sequences (||Mn|li q)nGN ^ 
(ll'^^nllip)neN C M are bounded, which we do in the two claims below. 

Before that, however, we remark that there exist constants ci > 0 and uq G Id such that 
J{un) ^ Cl and \ J'{unUn)\ ^ min I ||nn||ig, ll'^^nllip} for all n G N such that n ^ uq; and since 
9 > p > 1, for all n ^ uq we have 

- ^J'{Un)Un ^ Cl + ^ ^ Cl + miu { ||Mn|li,g , H^nHi^p }• (9) 

Claim 1. The sequence (||Mn|li q)neN C M is bounded. 


Proof of Claim 1. We divide our analysis into cases that mirror the dehnition of the nolinearity 
g. If |x| > R and f{t) > a{x)\t\P~H/k, then 

G{x,Un)dx = - [ g{x,Un)undx, 




P 


and this implies that 

J{'^n) ~J {'^n)'^n ( ~ ~ ] ||^n|| i „ • 

p \Q P J 

Combining inequalities (9) and (10) we conclude that 


( 10 ) 


1 1 

-Min 

q p 


n-\\l,q ^ Cl + , 


So, in this case the sequence (||n„||^g)„gN C M is bounded, say ||nn|lig ^ Cq for every n G N. 

If \x\ ^ i? or if |a;| > R and f{t) ^ a{x)\tf'~H/k, the boundedness of the sequence can 
be proved using the same ideas as that of the previous case with some minor changes. This 
concludes the proof of the claim. □ 

Claim 2. The sequence (||n„||^p)ngN C M is bounded. 

Proof of Claim 2. We also divide our analysis into the same cases. If |a:| > R and f{t) > 
a{x)\tf‘~‘^t/k, then we have 


J{'^n) {'^ri)'^n ^ ( 

9 \p 




1 
'9 

1 1 
p 9 

(P- 1) 
kp 


\u 


n\\l,p 


\U 


n\\l,p 


\u. 


n\\l,p 


+ n 


1 1 

q 9 

1 1 

p 9 

ll<? 

n\\l,q 


\U. 


nWl^q 


\U. 




1 

kp 

1 

kp 


a(x)|M„|^da; 


\\P I II 11? 


(11) 


Combining inequalities (9) and (11) and using Claim 1 we obtain 


(P- 1) 
kp 


\u 




^ Cl + llu 


nWi^p ■ 
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This means that in this case the sequence (||Mn|li p)nGN C M is bounded. 
If |a;| ^ i? or if |a:| > R and f{t) ^ a{x)\t\^~‘^tfk, then 


G{x,Un)<ix + - I g{x,Un)Un<^X ^ Q. 

u . -ON 


Hence, 


J(,'^n) qJ (^n)^n 

»(---) \K\K 


1 1 

q 6 


u 


nWl^q 


> 


1 1 

p 6 


I IIP I II 11^ 

Il,p IIII 1,( 


f G{x,Un)^X+ - / g{x,Un)Undx 
_ RJV u Jrjv 

G{x,Un)<^X + \ I g{x,Un)Undx 


e 


1 


^ ^ i + 11 “' 


<? 

>'*111,1} 




(P- 1) 
kp 


I IIP I II \\Q 

l“>*lll,p “1“ 


( 12 ) 


Combining inequalities (9) and (12) we get 

1 , 


— Ur 


k 


ll,p ^ Cl + ||u„||j^p . 


This means that also in this case the sequence (||un||^p)„eN C M is bounded. This concludes 
the proof of the claim. □ 


Using Claims 1 and 2 we deduce the proof of the lemma. 


□ 


The following result shows that the functional J verihes the Palais-Smale condition. 

Lemma 2.4. Suppose that the potential functions a, b verify the hypotheses {Pi), {P 2 ), and (P3) 
and that the nonlinearity f verifies the hypotheses {fi), (/2), (/s), and (/4). Then the Palais- 
Smale condition is valid for the energy functional J. 


Proof. Let {un)n&N C P be a Palais-Smale sequence at the level c; this means that 

J{un) c and J'{un) —t 0 

asn ^ oc. By Lema 2.3 this sequence is bounded. Then there exist a subsequence of {un)n&N C 
E, which we still denote in the same way, and there exists a function u ^ E such that Un ^ u 
weakly in P as n —?■ -|-cxd. 

For each e > 0, there exist r > R> 1 such that 


u\^ dx 

Let ?7 = ?7r G C°°(P^(0)) be a cut off function such that 0 ^ ^ 1, with r; = 1 in P2r(0) ^^id 

also |V? 7 | ^ 2/r for all X e M^. Since the sequence (un)neY C P is bounded, it follows that the 
sequence {pUn)n&N C P is bounded also. Therefore, J'{un){pUn) = o„(l), that is. 




2 (2^ - l)^/^a;^ (1 - 


/ H c 

’ r^\x\^2r 

f I 

' r<|a;|^2r 



+ 


/ 

JR 


^ Un ■ 'V (pUn) dx-\- / a{x)\u 


IVUnI'' Un ■ V{pUn) dx + 


^\P ^Un{pUn)dx 

b{x)\u\f~'^Un{pUn) dx 


g{x,Un){pUn) dx + o{l). 


( 14 ) 
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The previous expression and the properties of the cut off function rj imply that 
/ -qlWUn\^ dx + / • Vr^dx + / T]a{x)\Un\^ dx 

J \x\'^r J \x\^r J \x\'^r 

+ / ?7|VMn|'^da:+ / \VUn\''~‘^Un'VUn ■ V'qdx + / r]b{x)\Un\''dx 

J\x\^r J\x\^r J\x\^r 

r]g{x,Un)Un dx + o(l). 


J\x\^r 

By the inequality ( 3 ), it follows that 


ax 


'\x\^r 


T]g{x,Un)undx ^ / T]—— |n„|^dx; 




k 


thus, we obtain 


/ ?7|VM„|^dx+ / r7a(x)|M„|^dx 

' \x\'^r J\x\'^r 


a[x} 


+ / g\\/Un\'^dx+ / r76(x)|n„|^ dx — / g ——|u 


'\x\'^r 


'\x\'^r 


'|x|^r 


k 


^ / |Vm„|^ V„||V? 7 |dx+ / iVn^l'' ^^HVr^l dx + o(l) 

J\x\'^r J\x\^r 

|VMnr“Vn|dx+ f dx| + o(l). 

L far^|x|sg2r yr^|x|sg2r J 


Subtracting the terms 

-f 

^ J\x\'^r 


If If 

g\Vun\^dx+— / g\Vun\^dx+— / r76(x)|M„|^ dx 


' \x\^r 


k 


\x\^r 


from the left-hand side of the previous inequality and grouping the several integrals, we deduce 
that 


_ . g\VuJPdx+ / ?7a(x)|M„|Pdx 

_ _ J\x\^r J\x\^r 

+ I r^lVn^l'^ dx-I- / r76(x)|M„|^ dx 

J\x\'^r J\x\'^r 

\Un\\VUn\^~^dx+ I \Un\\VUn\'^~^ dx\ + 0{1). 

^ L^r^|a;|^2r Jr^\x\^2r 



Now we use Holder’s inequality to get 
[ |m„| |Vm„|^"Mx ^ 


' r^|a:|^2r 


\Un\^dx ) 

r^|a:|^2r / 


< 


r^|a:|^2r 





iVn^rdx 



p-i 


And in a similar way, we obtain 


/ |m„| Mx ^ dx I llwnll'^ 

^r^|x|^2r V'-'r^|a;|^2r 
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By the compactness of the embedding W^’^{B 2 r\Br) ^ U’{B 2 r\Br), we infer that Un ^ u 
strongly in LP{B 2 r\Br) as n —)■ oo. Since {T]Un)nm C fl it follows that 


limsup ( 1 — — 


^ - lim sup 

^ n—>-oo 



\x\^r 


ri\'Vun\^dx+ / 770(0:) dx 


'\x\^r 


+ / 77 |VMn|'^dx+ / r76(x)|'Un|'^ dx 


' \x\^r 


' \x\'^r 


[ iu„rdx) '"\\u4^-^ + ( 




\ 1/9 

lUnl'^dx] llWnir”^ 


-{(I |u|^dxV''||uf f 

^ 1 \J r^\x\^2r J V 


h^|x|^2r 


\ 1/9 

i«rdd ii«ii- 


(15) 


Applying Holder’s inequality once more and denoting the volume of the unitary ball by 
I Hi (0)1 = cuat, we obtain 


\u[ 


' r^|x|^2r 

And in a similar way, we obtain 

\ 1/9 


(( 2 « - 


r^|3::|:^2r 


\ i/p* 
|m|^* dx j 


\u\ 


dx) A ((2''-l)(x„rq‘"'(^ 


\u 


' r^|a;|^2r 

Substituting inequalities (16) and (17) in (15), we get 


r^|a:|^2r 


\ 1/9* 
dx] 


(16) 


(17) 


lim sup ( 1 — — 


'\x\'^r 


r7|VMn|^dx+ / r7a(x)|M„|^ dx 


'\x\'^r 


+ / r7|VMn|'^dx+ / r76(x)|Mn|'^ dx 

J\x\^r J\x\'^r 

( ~ \ i/p* 

iMi^*dx] 


«2((2''-l)a.„)‘''"<! 


' r^|a:|:^2r 




\ 1/9* 

|M|^*dx) 




(18) 


' r^|a:|^2r 

In particular, since 77 = 1 outside the ball of radius 2r, by inequalities (15) and (18) we obtain 


limsup ( 1 — 



^2((2^-l)a;,)^^^V 


' r^|x|^2r 

N 

Therefore, by inequalities (13) and (19) it follows that 


|VM„|^dx+ / a(x)|M„|^dx 

J |x|^2r 

VMn|'^dx+ / 5(x)|M„|^dx 

J\x\'^2r 

\ l/p‘ 

|M|^*dx| 

r^|a:|^2r / 

\ 1/9* 

I«|.-dx Ii«r- 


(19) 


lim sup 

n—>-oo 



|VMnrdx+ / a(x)|M„|^dx 


|a:|^2r 


J \x\^2r 

|Vun|^dx+ / 5(x)|M„|'^dx 

J\x\'^2r 


< e. 


( 20 ) 
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Combining inequalities (14) and (20), we deduce that 

limsup / g{x,Un)un<^x = (21) 

n^oo J\x\'^2r 

Now we use the dominated convergence theorem together with the fact that g has subcritical 
growth to infer that 


limsup / g{x,Un)undx = / g{x,u)udx-, 


( 22 ) 


n^oo J\x\^2r 


' |a:|^2r 


and since / g{x, Un)Un dx < oo, by the choice of r > i? > 1 and from equalities (21) and (22), 
we obtain 


lim / g{x,Un)Undx = / g{x,u)udx. 


(23) 


It remains to show that the norm sequence (||Mn||)neN C M is such that ||m„|| ||m|| G M as 

n ^ oo. Using Holder’s inequality and making some computations, it follows that 


o(l) = {J'{Un) - J'{u)) {Un - u) 




\VUn\^ dx 


(p-l)/p 


\Vu\Pdx 


{p-i)/p 


> X < 


+ 


a{x)\unf dx 


{p-i)/p 


+ 


a(x)|M|^da: 
(g-i)/9 


{p-i)/p 


> X < 


IVUnl^ dx 


+ 


iVulMx 


h{x)\Un\^ dx 


{9-l)/9 


(9-i)/g 


> X < 


h{x)\u\^ dx 


(9-l)/9 


> X < 


\ 1/p 

|Vu„|^dxj 

r \ 1/p 

iVul^dx 


\ 1/p 

a(x)|M„|^dx I 

r \ l/P 

a(x)|M|^dx I 

S.N J 

\ 1/9 

I dx j 
[ iVulMx) 

Irn j 

\ 1/9 

h{x)\Un\'^ dx j 


\ 1/9 

6 (x)|M|'^dx j 


- {g{x,Un) - g{x,u)){un-u)dx. 

Jrn 

We remark that all the terms between curly brackets in the previous expression have the same 
signals; therefore, by the limit (23) we get 


n—>-CXD 

and also 


lim / |VMn|^dx= / iVul^dx, lim / a(x)|'Un|^dx = / a(x)|'u|^dx, 

JRJV JRJV n^oo Jgjv JrJV 

I 

lim / |Vun|^dx= / iVul^dx, lim / 6(x)|u„|^dx= / 6(x)|u|^dx. 


This implies that 
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Moreover, Un ^ u weakly in as n —?■ oo; and finally, Un ^ u strongly in E' as n — )■ cxd. For 
the details, see DiBenedetto [23, Proposition V.ll.l], □ 

Lemma 2.5. Suppose that there exists a sequence (Mn)neN C E and a function u E E such that 
Un ^ u in E and J'{un) ^0 as n ^ oo. Then there exists a subsequence, still denoted in the 
same way, such that —)■ Vu a. e. in 

Proof. See Assungao, Carriao, and Miyagaki [ 8 ] or Benmouloud, Echarghaoui, and Sbai [13]. □ 

Using Lemmas 2.1, 2.2, 2.3, 2.4, and 2.5 we conclude that there exists u E E which is a critical 
point for the functional J. Moreover, this critical point is a positive ground state solution to 
the auxiliary problem ( 6 ), that is, J(n) = c > 0 and J'{u) = 0 . 


3. Estimate for the solution to the auxiliary problem 


In this section we show that the solution to the auxiliary problem ( 6 ) obtained in the previous 
section verihes an important estimate. To do this we use several lemmas. 


Lemma 3.1. For R > 1, every positive ground state solution u to problem ( 6 ) verifies the 
estimate 


u 


IIP 

lli,p 



dkp 

p — 1 


Proof. Combining inequalities ( 8 ), (11) and (12), it follows that 

{\Mi,P + \Mlg} ^ = J{u) = c^d. 

The conclusion of the lemma follows immediately. □ 


We remark that the boundedness of the norm of the ground state solution to problem ( 6 ) 
shown in Lemma 3.1 depends only on the potential functions a^o and boo, on the nonlinearity 
/ and on the constant 6] it is independ of the constant R> 1. 

The next lemma is a crucial step to establish an important estimate involving the norm of the 
solution to the auxiliary problem ( 6 ) in the space L°°(M^). To prove it we adapt the arguments 
by Alves and Souto [4]; see also Gilbarg and Trudinger [27, Section 8 . 6 ], Brezis and Kato [16], 
Pucci and Servadei [34], and Bastos, Miyagaki, and Vieira [10]. 

Lemma 3.2. Suppose that p,r E M. verify the inequality pr > N. Let H\ x M —)■ M 
be a continuous function such that \p[{x,s)\ ^ h{x)\s\^~'^s for all s > 0 with the function 
h: —)■ M so that h E and let A, B -. M. be nonnegative functions. Suppose also 

that V E E C. D^’P{R^) fl Zl^’^(M^) is a weak solution to the problem 

—ApV — AqV + A{x)\v\^~'^v + B{x)\v\‘^~‘^v = E[{x,v), xeM.^. (24) 

Then there exists a constant Mi = Mi{N,p,q,r, ||h|| 2 ,r(]RiV)) > 0, which does not depend on the 
functions A and B, such that 

^ Mimax{\\v\\Lp*(j^N), K, KLy,l} , 
where K and are defined by (28) and by (29), respectively. 

Proof. Let /9 > 1; for every m G M we dehne the subsets 

Ajn = {x e 1 < \v{x)\^~^ ^ m}; 

Bm = {x e |x(x)|^“^ > m}; 

Cm = {x e |x(x)|^“^ ^ 1}. 





EXISTENCE RESULT FOR (p-g)-LAPLACIAN WITH VANISHING POTENTIALS 


13 


We also define the sequence of functions (nm)mGN C n by 

{ \v{x)\P^^~^'>v{x), if X e Am] 
mPv{x), ii X e Bm] 

if X e Cm- 

It is easy to verify that for every x e we have Vm{x) ^ max ||x(x)|^^^~^^’'’^, |x(x)|^*'^~^^’'"^|. 
Additionally, simple computations show that 

({p{l3 - 1) + 1) Vv(x), if X e Am] 

Wvmix) = < mPVx(x), if X e Bm] 

[ {q{(3 — 1) + 1) |x(x)|'^^^“^^ Vx(x), if X e Cm,- 

Furthermore, (xm)meN C E. Indeed, 

f a(x)|xmrdx^ f a(x) dx + f a{x)\v\^~^vrnF^^~^'’'^^ dx 

JAm JBm 

+ / a(x) (|x|^“^x) dx 

JCm 

^ / a(x)|x|^“^x dx <+ 00 . 

JlR^ 

And in a similar way, we have 

/ 6(x)|xm|'^, dx = / &(x)|x|'^“^x dx <+ 00 . 

Multiplying both sides of the differential equation (24) by the test function Vm and integrating 
the left-hand with the help of the divergence theorem, we deduce that 

[ |Vx|^’-Vx-Vx^dx+ [ |Vx|'^-Vx-Vx^dx 

+ / A(x)|x|^“^XXm dx + / i?(x)|x|^“^XXm dx 

Jr^ Jr^ 

= / H{x,v)vmdx 

JrN 

Using the dehnition of the function Vm, we obtain 

(p(^-l) + l)( [ |Vx|P|x|P(^-^Mx+ [ |Vxnx|P(^-^)dx 


V V -Attj, */ -Attj, 

(g(^-l) + l)|/ |Vx|P|x|''(^-^Mx+ / |VxHx|'?(^-^)dx 
= / |Vx|^“^Vx • VXm dx + / |Vx|'^“^Vx • VXm dx 


-mP< |Vx|^’dx+ / iVxIMx 

V Bm Bm > 

^ / I Vx|^“^Vx • VXm dx + / A(x)|x|^“^XXm dx 

Jr^ Jr^ 

+ / I Vx|'^“^Vx • VXm dx + / i?(x)|x|'^“^XXm dx. 

Jrn Jrn 

Now we define another sequence of functions (tXm)meN C U by 

|x(x)|^“^x(x), lixeA^UU^; 


(25) 


Wm{x) = 


mv{x), 


if X G 
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{I3\v{x)\^ ^Vv{x), lixeA^UCm] 


Direct computations show that 

VWm{x) = <, 

I mVu(x), if x G Bm- 

Using the hypothesis 2^q^p<N,we obtain 

[ \'VWm\^dx+ [ A{x)\Wm\^ dx — [ I ■ VUm dx— 


A{x)\v\^ ‘^VVmdx 


+ / |Vt<;m|'^dx+ / B{x)\Wm\‘^ dx — / |Vu|'^ ‘^'V V ■ V Vm dx — / B{x)\v\‘^ ‘^VVmdx 

JrN J]RiV jTg^N 

[ \Vv\P\v\P^^-^Ux + (3P [ \Vv\^\v\^^^-^Ux 

UC7m J A. m UCm 

— (p(/9 — 1) + 1) I [ (ia; _|_ f 


+ f B{x) (lul"^ - dx+ f A{x) (|u|P^ - \v\P+A0-^)) dx 

Am •J Cm 

+ [rrA — tuF) f 5(x)|u|'^dx 

^ Bm 

-(g(/3-l) + l)( / \Vv\P\v\‘^^^-^Ax+ f |Vunu|''(^-^Mxl 
UCm JCm J 

+ (m'^ — mP) f |Vu|'^dx. 

^ Bm 

And after we get rid of the non positive terms, we can regroup the expressions to obtain 

/ |VtCmrdx+ / A{x)\WrrA dx + / |VtCm|^dx+ / i?(x) dx 

Jr’^ JrN JjiN J^N 

= {(3P-ip{P-l) + l)) f \Vv\p\v\p^^-^Ax + (3p f \Vv\P\v\P^f^-^Ax 

'J Am ^ Cm 

+ C‘^-W-1) + 1)) f |Vu|''|u|''(^-^Mx + /3'' / |VuHu|''(^-^Mx 

Cm ^ Am 

+ / I Vu|^“^Vu • VUm dx + / A(x)|u|^“^UUm dx 

JlR^ Jr^ 

+ / |Vu|'^“^Vu • VUm dx + / i?(x)|u|'^“^UUm dx 

JrN JrN 

So, using inequality (25) we deduce that 

|VM;mrdx+ [ A(x)|M;m|^dx + [ |VM;m|'^dx+ [ B{x)\Wm\'^ dx 

~" J RN Jrn 

I Vu|^“^Vu • VUm dx + / A(x)|u|^“^UUm dx 

Jrn 

+ / |Vu|'^“^Vu • VUm dx + / i?(x)|u|'^“^UUm dx 

Jrn Jrn 

+ pp I \Vv\P\v\P^^-^Ax + j |Vunu|'?(^-^)dx. 

Now we estimate some integrals that appear in the previous inequality. First, by dehnition 
of Arr, we have 



[ |VuF|uF^^“^Mx = [ __ 


Vu • VUm dx 
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^ / |Vt|^ V ■ VVm dx + / A{x)\vf ‘^VVmdx 


+ / |Vt|^ ‘^'VV ■ VVm dx + / B{x)\v\'^ ‘^VVmdx. 
In a similar way, by definition of Cm we have 

^ [ |da; + f A{x)\v\P-hvmdx 

aRJv JMiv 

+ / I Vn|'^“^Vn • da; + / i?(a:)|n|^“^nnm da:. 


'Crr 


Using these inequalities we deduce that 

[ |Vu;mrda:+ [ A{x)\wmf dx + [ \Vwm\'^dx+ [ i?(a:)|u;m|'^ da: 

JRJV JjjAT JjlN JrJV 




I3P 


g(/3-l) + l 



|Vn|^ ^Vn-Vnmdx+ / A(x)|n|^ ^numda; 

+ / I Vn|^“^Vn • Vum + / i?(a:)|n|^“^nnm dx 



|Vn|^ ^Vn-Vnm,dx+ / 74(x)|n|^ dx 

Jrn 

+ / |Vn|^“^Vn • Vxm dx + / i?(x)|n|^“^nnm dx 


^ 2/3^ 


= 2/3^ / if(x, n)nmdx. 


Using the Sobolev inequality (7) and the hypothesis i7(x, s) ^ h(x)|s|^ we obtain 




AttiUOttt, 


\ p/p* / 

dx j ^ ( 





\ p/p* 

tCmr* dx j 

|Vtam|^dx+ / a(x)|M;mrdx 
+ / |Vtam|'^dx+ / 6(x)|M;m|^dx 



H{x,v)vmdx ^ 2S(3P / h(x) |n|^ ^n^dx 

/i(x) adx + f h{x) v mPy dx 

J Bm 

+ f h(x) |a|^^^~^^adx 

ac^ 

h(x)|a|^^dx+ / h(x)|a|^dx>, 


where in the last passage we used the definitions of the functions Vm and Wm, together with the 
facts that in Bm we have |tam|^ ^ and in Cm we have |u|p+i(/3-i) ^ l-ylP. 

Passing to the limit as m —)■ cxd and using Lebesgue’s dominated convergence theorem, it 
follows that 
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Applying Holder’s inequality to both terms on the right-hand side of the previous inequality, 
we obtain 


h{x) \v\^^ dx ^ 


p/3 


and 


hence 


h{x) \v\^ dx ^ ||h||ir-(KiV)||n 


I J^pr' 


I P/3 

ILP*P{S.^) 


^ 2S||ft||i,|MA.)^'’ |max {||k||"V'(ii») 

= Cf(3Pmax\\\v 


Lpr 


, 1 i +max<' 






'(IRJV)) l| I ) 


where we used the notation Cf = Cf{N,p,q,r, ||h||p^r(]RiV)) = 45'||/i||p^r(]RiV) > 0. 
Writing (3 = for j G N we deduce that 

lklliP*. 3 (RiV) ^ max |||n||^p,,V'(RiV), max | l||. 

Choosing a = p*/pr' > 1, from inequality (26) with j = 1 we obtain 


\LP 


^ max|||n 


\LP 


|V' 


,max<i 




and from inequality (26) with j = 2 together with the previous inequality we obtain 


max 111n 11 lp* (r^) , max 111n 11 , 111 , 


^ C<y- ^2/a^ 
< (g^/cr+l/cr^ 


max < n 


|iA’ 

I J^pr> 


,1 




X max 


|1/(T 

I l^pr' 


{(G'VV.)-',!}, 

max I\l|max| 

Proceeding in this way, for j G N we obtain 

lkiyp*.3(RiV) ^ Cyahmax{||'i;||^p*(RiV),A'j,A:jLj} , 
where Sj = l/a + l/a^ + • ■ ■ + 1/a^; tj = 1/a + 2/a'^ + ■ ■ ■ + j/cr^] 

|maxi5i5,_i {C,‘V ‘S 1} , if/^2; 

and 




|iA= 

I l^pr' 


Lj = max <1 ||n 


ll/a 

Hpr 


Since a > 1, we have hmj_j,ooSj = l/(o' — 1) and limj_j,ootj = a/{a — 1)^; hence, 

(^l/(<x-l)^, 

\c\/^a^/^y^ ifCi>l; 


3 1 ^ 


J -/00 


(26) 


(27) 


(28) 
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and 


lim Lj = Ly = 

j-foo 


1 , 


if llnl 


I l^pr' 


l^pr' 


^ 1 ; 


if 11 ^ 11 i/P’’' ^ ^ ■ 


(29) 


Using the fact that v E E d D^’P(R^) applying Holder’s inequality we deduce that 

Ly < +CX 0 . 

Finally, passing to the limit as j —)■ cxo and using inequality (27) we obtain 


= lim ||n| 

j^oo 


Lp*it3 


(30) 

□ 


= Mimax , J7, J7L„, l|, 

where Mi = Mi{N,p,q,r, ||h|| 2 ,r-(KiV)). This concludes the proof of the lemma. 

Lemma 3.3. For every R > 1 there exist a constant M 2 = M 2 {N,p,q,r,aoo,boo,d,Co) such 
that any positive ground state solution u G D^’P{R^) n to the auxiliary problem (6) 

verifies the inequality 

^ M 2 . 


Proof. Consider R> 1 and let u G fl be a positive ground state solution to 

the auxiliary problem (6). Now we dehne the function H: x M —)■ M by 


Fl{x^ t) = 


f{t), if \x\ ^ i? or if |a;| > R and f{t) ^ ^ ^ |t|^ 


0, if |a;| > R and f{t) > 


a(x} 


\t\p-H. 


We also dehne the functions A, B : —)■ M by 


a{x), if |a;| ^ i? or if |a;| > R and f{u{x)) ^ 

H(x) y \ a(x) 

1 — — j a{x), if \x\ > R and f{u{x)) > —^ u{x), 

and B{x) = b{x). 

Considering these functions and using n G 77 as a test function, we have 


a{x) 


u{x); 


0 = / |Vm|^ ^Vn-Vnda;+ / H(a:)|M|^ ‘^uv dx 

Jrn J]^n 

+ / I • Vn dx + / B{x)\u\'^~‘^uv dx 


H{x, u)v dx 




|Vm|^ ‘^Vu-Vvdx+ / a{x)\uf’ “^uvdx 


+ / |Vm|'^ ^VM-Vnda;+ / b{x)\u\^ ‘^uvdx— / g{x,u)vdx. 


From hypothesis (/i), for |t| small enough we have |i7(x,t)| ^ |/(^)| ^ from 

hypothesis (/ 2 ), for |t| big enough we have |i7(a;,t)| ^ |/(^)| ^ C 2 |t|'’'“^ with r G {p,p*). 
Combining both cases we obtain |i7(x,t)| ^ ^ Co|t|^ for every t G M’*' and for every 

r G {p,p*). Then, it follows that |i7(a;,M)| ^ cq\u{x)\'^~^\u{x)\p~^ = h{x)\u{x)\P~^, where we 
dehne h{x) = co\u{x)\'^~p. 

Direct computations show that h G L'’(M^) for r = p*/{r — p). Indeed, 



\h{x)f dx ^ 



uF* dx 
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^ SP*/P 


P*/p 


^ (^0 


■*/(t-p) 


gP*/p 



\Vu\Pdx 
|VM|^’da; + 


p*/p 


a{x)\u\^ dx 
|VM|'^dx+ [ b{x)\u\'^ dx 


< 


^*/i'r-p) QP*/p 


I IIP I II 11^ 

Ml,p+\Mlq 


P* /P 


< + 00 . 


In this way, any positive ground state solution u G fl to the auxiliary 

problem (6) verifies the hypothesis of Lemma 3.2. Concluding the argument, from inequality (7) 
and from Lemma 3.1 we have 


(K^) 




I IIP I II \\Q 

I«II1p+ ll“llr 


y/p ^ f Sdkp Y^^ 


\P-i) 

Finally, combining estimate (30) with the previous inequality we obtain 

||w||l°°(kjv) ^ Ml max I ||M||^p*,„iVN, K, KL„, 1 


^ Ml max { 1 


= Mo 


□ 


where M 2 = M 2 (iV, p, q, r, a^o, boo, d, Cq)- The lemma is proved. 

Lemma 3.4. Suppose that R> 1 and let u G fl be a positive ground 

state solution to the auxiliary problem ( 6 ). Then u verifies the inequality 

pi{N-p)/{p-l) 

uix) ^ M 2 —rrr ;^—777 -tt 

\x\^^~piSp~d 

for every \x\^ R> 1. 

Proof. Given Rq ^ R > 1, we define the function v: M^\{0} —)■ M by 


v{x) = M 2 


R 


{N-p)/{p-l) 


|2;|(Y-p)/(p-1) ■ 

By hypothesis, u G D^’P{R^) fl is a positive ground state solution to the auxiliary 

problem (6); therefore, we can apply Lemma 3.3 to deduce that ||m||l°°(r^) ^ M 2 . This implies 
that if |a;| = Rq, then ||M||Loo(RiV) ^ v{x). Now we define the function w. M^\{0} —)■ M by 


w{x) = 


0 , 


if |a:| ^ Rq] 


{u — n)"*", if |a:| ^ Rq. 


In this way, w G D^’P(R^) fl moreover, w E E because u,v E E. 

To complete the proof of the lemma we will show that {u — n)'*' = 0 for |a;| ^ Rq. To 
accomplish this goal we use the hypotheses on the potential functions a and 6; we will also use 
the function w E E a.s a. test function to obtain 

/ • Vtcda; + / \'Vu\'^~‘^'Vu ■'Vwdx 


g{x, u)w dx 


a{x)\uf’ "^uvcidx— / b{x)\u\^ "^uwdx 


/R^\Bi{p(0)A/(t)^a(x)|t|P '^t/k 


f{u)w dx + 


a{x) 


\uF '^uwdx 


(0)Af{t)>a{x)\t\P ‘^tjk 
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/ a{x)\u\^ “^uwdx — / h{x)\u\'^ "^uwdx 




a[x) 


lR^\BRg{0)Af{t)i:a{x)\t\P-2t/k k 


ImP ^uw dx 


+ 


f 

'R^\Bii^iO)Afit)>a{x)\t\P-^t/k k 


Mp ‘^uwdx 


/ a{x)\u\^ “^uwdx— / b{x)\u\'^ “^uwdx 

Ir^\Br^{0) Jr^\Br^{0) 


Ir^\Br^(0) 


a{x)\u\^ '^uwdx 


IR’^\Br.{0) 


b{x)\u\'^ '^uw dx 


^ 0 


(31) 


because u is a positive function and tc is a nonnegative faction, while k > 1. 

Using the radially symmetric form of the operator A^n, we have 

[ iVnr-^Vn-V0da; = O 

Jr>^\Br^{0) 

for m G {p, q} and for every function (f) ^ E. Therefore, 

/ |Vnp“^Vn • Vtc dx + / |Vnp“^Vn • Vtc dx 

JRA! JRN 

= / |VnP“^Vn • Vtcda; + / |VnP“^Vn • Vtcdx 

Jr^\Br^{0) JrN\Br^{0) 

= 0. (32) 

Defining the subsets 

A = {x E |x| ^ Rq and u{x) > n(a:)} 

and 

B = {x & |x| < Rq or u{x) ^ n(x)}, 

we have w{x) = u{x) — v{x) for a; G A and w{x) = 0 for a; G -B. Using inequality (31) and 
equation (32) we get 

0^ / |VmP“^Vm • Vtcda; + / | • Vtc da; 

JRJV jRiV 

- [ \Wv\P-^Vv -Vwdx- [ iVnp-^Vn • Vwdx 

JRJV jRiV 

= [ [I Vnp-^Vn - |VnP"^Vn] ■ (Vn - Vn) da: 


+ [|Vnp-Vn - |Vnp-Vn] ■ (Vn - Vn) da;. 
J A 


(33) 


Denoting by (•,•): x —)■ M the standard scalar product, given p ^ 2 there exists a 

positive constant Cp G M’*' such that for every x,y E it is valid the inequality 

(|xP"^x - \y\^~^y, x-y)^Cp ||x - i/f (34) 

For the proof, we refer the reader to Simon [36]. From inequalities (33) and (34) it follows that 


iVwPdxT / |VM;Pdx= / |Vn-VnPdx / iVn-VnPda; 

PrJV Ja j a 
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^ ^ L - |Vvr"^Vu] • (Vm - Vv) da; 

Ja 

+ c-i [ [I - |Vv|'^-Vu] ■ (Vm - Vw) dx 

J A 

^ 0 . 

From this inequality we deduce that each term on the left-hand side of the previous inequality 
must be zero, that is, w is constant in But we already know that w{x) = 0 in the ball 
Bro{0); therefore, w{x) = 0 for every x G This implies that {u — n)"'' = 0 for |a:| ^ Rq and 
u{x) ^ v(x) for every x G The proof of the lemma is complete. □ 


4. Obtaining the solution of the original problem 


In this section we hnally show that the solution to the auxiliary problem (6) obtained in 
section 2 is in fact a solution to problem (1). 


Proof of Theorem 1.1. From Lemmas 2.3 and 2.4, the auxiliary problem (6) has a positive 
ground state solution u G Pi To accomplish our goal we need to show that 

for every x G 5^(0) the function u verifies the inequality 

f^u) ^ ^ \ur\. 

From Lemma 3.4 and by the hrst inequality in (2), if |a;| ^ i?, then 


f{u) 


\u 


\p-2 


U 


< Co 


\u 


\p*-2 


\U 


\p-2 


^ Co< M2 




-1)\(N-P)/P ^ P*-P 


(|a;|p/(p-b) 




= CoM|-" 


\^\v'^/(p-l) ■ 


Now we define the constant 

A* = cokMf-P. 

Considering A ^ A*, it follows from the hypothesis (P3) that 

f{u) ^ A* ppV(p-i) ^ A ppV(p-i) ^ a{x) 
\u\P-^u ^ k |a;|pV(p-i) k |a;|p^/(p-b k 
The proof of the theorem is complete. 


□ 
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